Tidy data is an alternative name for the common statistical form called a ''model matrix'' or ''data matrix''. A
data matrix
A Data Matrix is a two-dimensional code consisting of black and white "cells" or dots arranged in either a square or rectangular pattern, also known as a matrix. The information to be encoded can be text or numeric data. Usual data size is fro ...
is defined in as follows:
A standard method of displaying a multivariate set of data is in the form of a data matrix in which rows correspond to sample individuals and columns to variables, so that the entry in the ''i''th row and ''j''th column gives the value of the ''j''th variate as measured or observed on the ''i''th individual.
Hadley Wickham
Hadley Alexander Wickham (born 14 October 1979) is a statistician from New Zealand and Chief Scientist at Posit, PBC (former RStudio Inc.) and an adjunct Professor of statistics at the University of Auckland, Stanford University, and Rice Un ...
later defined "Tidy Data" as
data set A data set (or dataset) is a collection of data. In the case of tabular data, a data set corresponds to one or more database tables, where every column of a table represents a particular variable, and each row corresponds to a given record of the ...
s that are arranged such that each variable is a column and each observation (or ''case'') is a row.
(Originally with additional per-table conditions that made the definition equivalent to the
Boyce–Codd 3rd normal form.)
Data arrangement is an important consideration in data processing, but should not be confused with the also important task of
data cleansing
Data cleansing or data cleaning is the process of detecting and correcting (or removing) corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate or irrelevant parts of the dat ...
.
Other relevant formulations include
denormalization
Denormalization is a strategy used on a previously- normalized database to increase performance. In computing, denormalization is the process of trying to improve the read performance of a database, at the expense of losing some write performance, ...
prior to machine learning modeling (informally denoting moving data to a "wide form" where all possible measurements are in a given row), and use of
semantic triples as intermediate representation (informally a "tall" or "long" form, where measurements about a single instance are spread across many rows).
References
{{Reflist
Data processing